Search for: All records

In this paper, we present a conformal prediction (CP) based method to evaluate the performance of a finger-printing localization system through uncertainty quantification. The proposed method emphasizes a standalone module that is compatible with any well-trained fingerprint classifier without incurring extra training costs. It provides rigorous statistical guarantees for revealing true labels in the fingerprinting multi-class classification problems with high efficiency. Uncertainty quantification of the predictions is accomplished by leveraging a small calibration dataset and a given error tolerance level. Three specific metrics are introduced to quantify the uncertainty of the CP-based method from the perspective of efficiency, adaptivity, and accuracy, respectively. The proposed method allows developers to track the model state with minimal effort and evaluate the reliability of their model and measurements, such as in a dynamic environment. The proposed technique, therefore, prevents the intrinsic label inaccuracy and the additional labor cost of ground truth collection. We evaluate the proposed method and metrics in two representative indoor environments using vanilla fingerprint-based localization models with extensive experiments. Our experimental results show that the proposed method can successfully quantify the uncertainty of predictions.